A smart method for spark using neural network for big data

نویسندگان

چکیده

Apache spark, famously known for big data handling ability, is a distributed open-source framework that utilizes the idea of memory to process data. As performance spark mostly being affected by predominant configuration parameters, it challenging achieve optimal result from spark. The current practice tuning parameters ineffective, as performed manually. Manual large space and complex interactions with among parameters. This paper proposes more effective, self-tuning approach subject neural network called Smart method using (SSNNB) avoid disadvantages manual has selected five different sizes test approach. proposed increased speed around 30% compared default parameter configuration.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

a new approach to credibility premium for zero-inflated poisson models for panel data

هدف اصلی از این تحقیق به دست آوردن و مقایسه حق بیمه باورمندی در مدل های شمارشی گزارش نشده برای داده های طولی می باشد. در این تحقیق حق بیمه های پبش گویی بر اساس توابع ضرر مربع خطا و نمایی محاسبه شده و با هم مقایسه می شود. تمایل به گرفتن پاداش و جایزه یکی از دلایل مهم برای گزارش ندادن تصادفات می باشد و افراد برای استفاده از تخفیف اغلب از گزارش تصادفات با هزینه پائین خودداری می کنند، در این تحقیق ...

15 صفحه اول

Forecasting Natural Gas Demand Using Meteorological Data: Neural Network Method

The need for prediction and patterns of gas consumption especially in the cold seasons is essential for consumption management and policy planning decision making. In residential and commercial uses which account for the bulk of gas consumption in the country the effects of meteorological variables have the highest impact on consumption.  In the present research four variables include daily ave...

متن کامل

A conjugate gradient based method for Decision Neural Network training

Decision Neural Network is a new approach for solving multi-objective decision-making problems based on artificial neural networks. Using inaccurate evaluation data, network training has improved and the number of educational data sets has decreased. The available training method is based on the gradient decent method (BP). One of its limitations is related to its convergence speed. Therefore,...

متن کامل

A New Method for Intrusion Detection Using Genetic Algorithm and Neural Network

    The article attempts to have neural network and genetic algorithm techniques present a model for classification on dataset. The goal is design model can the subject acted a firewall in network and this model with compound optimized algorithms create reliability and accuracy and reduce error rate couse of this is article use feedback neural network and compared to previous methods increase a...

متن کامل

A New Method for Intrusion Detection Using Genetic Algorithm and Neural Network

    The article attempts to have neural network and genetic algorithm techniques present a model for classification on dataset. The goal is design model can the subject acted a firewall in network and this model with compound optimized algorithms create reliability and accuracy and reduce error rate couse of this is article use feedback neural network and compared to previous methods increase a...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: International Journal of Power Electronics and Drive Systems

سال: 2021

ISSN: ['2722-2578', '2722-256X']

DOI: https://doi.org/10.11591/ijece.v11i3.pp2525-2534